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SPECIFICATION 



TO ALL WHOM IT MAY CONCERN: 

BE rr KNOWN, that I, Michael J. Freeman, Ph.D., and Craig UUman, citizens of the United 

States and residents of the State of New York, have invented certain new and useful improvements in: 

DIGITAL INTERACTIVE SYSTEM FOR PROVIDING 
FULL INTERACTIVITY WITH LIVE PROGRAMMING EVENTS 

of which the following is a specification. 



PATENT APPLICATION 
Attorney Docket No. 5038.25 
Express Mail Label No. EF084879952US 



DIGITAL INTERACTIVE SYSTEM FOR PROVIDING 
FULL INTERACTIVITY WITH LIVE PROGRAMMING EVENTS 

RELATED APPLICATIONS 

This is a divisional application of Serial No. 08/815,168 filed March 11, 1997, which is a 
continuation-in-part of application Serial No. 08/598,382, filed February 8, 1996, which is a 
continuation-in-part of application Serial No. 08/443,607, filed May 18, 1995, which is continuation- 
in-part of application Serial No, 08/166,608, filed December 13, 1993, now abandoned, which in turn 
is a continuation of application Serial No. 07/797,298, filed November 25, 1991, now abandoned. 
Each of the above-identified patent applications or patents is hereby incorporated by reference as if 
fully disclosed herein. 



BACKGROUND OF THE INVENTION 

Interactive video and audio presentation systems are currently being introduced into the 
entertainment and educational industries. A prominent interactive technology that has been applied 
successfully in these industries is based on providing interactivity in a one-way system through the 
provision of multiple time-synched parallel channels of information. For example, commonly owned 
Freeman et al. patents, U.S. patent nos. 4,264,925 and 4,264,924, which provide both audio and 
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video interactivity, disclose interactive television systems where switching among multiple broadcast or 

cable channels based on viewer selections provides an interactive capability. 

These systems have been enhanced to include memory functions using computer logic and 

memory, where selection of system responses played to the viewer are based on the processing and 
5 storage of subscriber responses, as disclosed in Freeman patent, U.S. patent no. 4,507,680. 
i;0 The benefits of providing interactivity through the use of different audio responses is disclosed in 

^ Freeman, U.S. patent nos. 4,847,698, 4,847,699 and 4,847,700. These television systems provide a 
m common video signal accompanied by several synchronized audio channels to provide content related 
H user selectable responses. The audio signals produce different audio responses, and in some cases, 

ift these are syllable synched to a first audio script and to the video signal (such as to a person or 

¥, y 

n character on a display), providing the perception that the person's or character's mouth movements 
match the spoken words. 

Interactivity is brought to the classroom in the Freeman U.S. patent no. 5,537,141. The 
distance learning system claimed in this application enhances the classroom educational experience 
15 through an innovative use of interactive technology over transnaission independent media. When an 
instructor, either broadcast live on video or displayed from videotape, asks a question, each and every 
student responds, preferably by entering a response on a remote handset, and each student immediately 
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receives a distinct and substantive audio response to his or her unique selection. The individualization of 
audio response from the interactive program is a major aspect of the invention. 

Individualization of audio is brought to the home based on the technology disclosed in Freeman 
U.S. patent no. 5,585,858. This system provides a program that can be watched on any conventional 
5 television set or multimedia computer as a normal program. But if the viewer has a special interactive 
% program box connected to the television, he or she can experience a fully functional interactive 

program. Each interactive viewer enjoys personalized audio responses and video graphics overlayed 
on the screen. The interactive program can be provided to television sets or to computers by cable, 
direct broadcast satellite, television broadcast or other transmission means, and can be analog or digital. 
lO However, what is needed is an interactive presentation system for providing true video, audio 

Jj^ and graphics interactivity with digital programs broadcast live. Such a system must efficiently package 
all the digital elements of the live interactive program at a centralized control studio and allow viewers at 
home to receive personalized interactive progranmiing. 

SUMMARY OF THE INVENTION 

15 In accordance with the invention, there is provided an interactive digital system allowing the 

viewer active participation in selecting digital video streams, associated with different camera angles, for 
example, and integrated audio and/or graphics segments. Further, Web pages from Internet Web sites 
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can be integrated into the program. The invention is particularly suited for the environment of live 
events, such as the broadcast of live sporting events. The viewer can appear to direct the camera shots 
by instantly changing among various camera angles, choose player interviews, or display associated 
statistical data on the team or players via graphics. In this manner, the system allows the individual 
subscriber to act as if he or she has control over how the program is directed and presented on their 
personal television set. In addition to selecting different camera angles, various audio options, closeups, 
slow motion, replays, graphics overlays, graphics or audio from Web sites, etc., are all possible. 
Further, games can be integrated with the live sports programming to increase viewer interest. 

Thus, viewers can customize the content of programs. The interactive digital programming of 
the present invention is particularly advantageous for viewing live sporting events. Viewers are not 
limited to selecting from multiple camera angles, but may also call up player statistics on demand, listen 
to selected player interviews, etc. Cameras can be focused on different segments of an event. 
Further, video options could include video replay, slow motion effects, isolation on a particular player 
or group, etc. Changes are seamless, thereby adding to the effect that the viewer is directing the 
television show just as a director now does from a control room. 

This "director" role by the viewer is possible due to the interactive technology of the present 
invention and also due to the digital compression and transmission scheme which allows for much 
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greater information throughput over a given bandwidth, allowing viewers to choose from angles that are 
already available but presently cut by the director. 

The digital interactive system is based upon branches which occur in the course of the full- 
motion video. Branches are real-time parallel paths that may be other full-motion video segments, 

5 graphics which are integrated into the video, audio segments, and/or retrieved Web pages which are 

5f integrated into the live event. 

,J Sometimes, the interactive digital system will act upon the viewer's response immediately; other 

yl times, it will utilize ACTV's unique "profiling" concept to act upon the response later. This technology 
: ^ enables the system to "remember" the viewer's responses and desires, and integrate them into the 
l||l video, audio, graphics and/or Web site information at a later point. For example, the viewer could 
O specify at the beginning of a football game to isolate the offensive quarterback of a particular team. 

Thus, whenever the team of choice is on offense, the video isolation of the quarterback is displayed to 
the viewer automatically. Or, based on how a viewer has selected camera angles, replays, etc., over 
the past five minutes, the system acts to mimic these selections at later times during the program. The 
15 system of the present invention "leams" from the viewer how they want to view the game, and thus, 
continues viewer selection sequences made earlier. 

At the source, the present invention comprises a plurality of video cameras, each of the video 
cameras relaying a different predetermined view of an event. The video signals corresponding to the 
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different cameras are forwarded to a central control studio. Further, one or more audio signals or 
graphic statistical overlays can be collected and sent to a central control studio. After receiving the 
video, audio, and graphics signals at the central control studio, these signals are digitized and 
compressed in digital video and audio compressors. These signals are then combined with special data 
codes into a "digital package," and subsequently, transmitted over a cable distribution system. The 
special data codes are the keys to unlocking the interactive potential of the program. 

The digital program signals are transmitted to a receive site by any suitable transmission means. 
Once received by a receive antenna, the digital program signals are passed along on a digital cable 
television distribution system to the viewer homes. Further, some other signals or commercials can be 
inserted at the local head end. The signals are received and processed in a digital cable box. 
Selections of the video, audio, graphics display and/or Web pages can be made as a function of 
immediate viewer entries, or to interrogatory responses presented at the beginning or during the 
program, or based on a prestored viewer profile. Once a decision is made to switch from one video 
option to another video option, the digital switch is performed seamlessly. 

As mentioned above, the program at predetermined times or immediately upon user entry can 
retrieve and branch to informative segments from Web sites. For example, a viewer watching a 
sporting event, through the system of the present invention, can receive a stream of Web pages which 
provide additional, specific information relating to a favorite player, team or perhaps the remaining 
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schedule for the sports team, as examples. In addition, users can take advantage of the two-way 

capabilities of the Internet to respond to polls or to link to additional sites. 

Another Internet-based application allows advertisers to speak more directly to consumers by 

directly sending Web pages to the consumer instead of merely displaying Web addresses in their 
5 commercials. The particular advertising information from Web sites can be targeted to viewers based 
J on the viewer profile, stored either in the digital set top box or at the cable headend. Alternatively, 

Web site access can be initiated by the viewer by simply clicking on the remote during the commercial. 
^ Thus, viewers have the capabiUty to individually select Web sites if they want more information from 

iJ, si 

1 advertisers, for example. 

lO The video programming and corresponding Internet pages can be viewed either on personal 

computers equipped with a television card on special digital cable boxes with stored interactive Litemet 
application software providing Internet access, or on digital television sets, all of which would utilize the 
specialized TV/Internet software of the present invention. 

The present invention also has applications for other types of programming. For example, 

15 viewers can direct the scenes of a murder mystery. Switching from one scene to another can be done 
seamlessly without noticeable effect on the viewer. Further, the present invention can be used for any 
kind of live or pre-recorded event. For instance, a music concert or a political convention can be 
enhanced in the manner of the invention. 
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Accordingly, a primary objective of this invention is providing an enhanced digital live program 
allowing the display to be tailored to the user's desires, choices or interests. 

It is an object of this invention to personalize and enhance live sporting events for the viewer. 

DESCRIPTION OF THE DRAWINGS 

Figure 1 is a diagram of the network and equipment for providing live digital programming. 
Figure 2 is a block diagram of an interactive digital cable box allowing seamless switching 
between video signals. 

Figure 3 is a block diagram of an alternative dual-tuner interactive digital cable box allowing 
seamless switching between video signals. 

Figure 4 is a block diagram of another alternative interactive digital cable box allowing seamless 
switching between video signals. 

Figure 5 is a time diagram showing a representation of trigger points and corresponding video, 
audio and/or graphics segments, one or a combination of which are selected for presentation to the 
subscriber immediately after the execution of the trigger point function. 

Figure 6 is a block diagram of an alternative embodiment of the interactive system including 
Internet access. 

Figure 7 is a block diagram of the two-way configuration of the system. 
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DESCRIPTION OF THE PREFERRED EMBODIMENTS 

The present invention is an interactive digital system 1 for producing a powerful personalized 
program allowing the home viewer an expanded set of programming options. Digital TV streams are 
put into digital packages made up of video, audio, data codes and graphics, and are used to provide 
personaUzed responses to viewer selections. Such responses can be further enhanced by allowing 
access to Litemet Web sites 170. In this manner, sports such as golf, football, baseball, basketball, etc. 
can now be watched with greater interest and involvement. However, even further enhanced 
interactivity is possible with the present invention due to the provision of various profiling and memory 
features. 

As shown in Figure 1, the present invention begins with the gathering of several possible video 
streams by way of cameras 100, strategically located at a sporting event 10, for example. Currently, 
many cameras 100 are employed at a sporting event 10. Super Bowl coverage, for example, typically 
encompasses 25 to 30 cameras. These live video streams can be integrated with recorded video 
streams which, for example, could include highlights from the current game or past games, player 
profiles, etc. To describe the components and operation of the present invention, the production and 
transmission of a live sporting event 10, i.e., football, is chosen to present the invention features. 
However, other applications can be appreciated by the reader, including several disclosed below. 
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A. The Control Studio 

The interactive broadcast program is prepared at the control studio 5 into digital packages. 
The control studio 5, as shown in Figure 1, allows a producer to create and introduce interactive 
elements during a live broadcast. In one preferred embodiment, the producer pre-records a set of 
interrogatories or instructions for the user. These interrogatories may include such questions as the 
following: 



SELECT THE CAMERA ANGLES/OPTIONS THAT YOU PREFER: 

OPTIONS: END-ZONE 

FIFTY YARD LINE 
FOCUS ON THE QUARTERBACK 
FOCUS ON THE DEFENSE 
FOCUS ON THE CHEERLEADERS 
PROVIDE HIGHLIGHTS 

WHO IS YOUR FAVORITE TEAM IN THE GAME? 

OPTIONS: BLUE TEAM 

RED TEAM 

WHO IS YOUR FAVORITE PLAYER IN THE GAME? 

OPTIONS: RED RUNNER 

BLUE QUARTERBACK 
BLUE DEFENSIVE END 
RED QUARTERBACK 
ETC. 
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ARE YOU INTERESTED IN RECEIVING STATISTICAL SUMMARIES? 

WOULD YOU ENJOY LISTENING TO PLAYER INTERVIEWS DURING THE 
GAME? 

Such interrogatories can be presented to the viewer at the beginning of the broadcast or 
scattered throughout the program. Interactive responses to such interrogatories would include video, 
audio and graphics personalized to the particular viewer. 

The preparation of the graphics for presenting such interrogatories occurs off-line at the control 
studio 5 using chyron or any graphics language. These interrogatories or instructions will ultimately be 
displayed to the home viewer, preferably in the form of graphics, to facilitate the interactive responses. 
The producer creates these graphic video slides of questions on a computer using the text editor and 
chyron. Associated with each question, the producer enters a number of possible viewer options. 
Then, the producer relates each possible user entry to one or more corresponding interactive 
responses. If the response is information from an Internet Web page, the producer will indicate the 
particular Uniform Resource Locator (URL) of the Web page. The producer sets a time stamp for 
when each particular question will appear during the program. As explained below in more detail, the 
viewer response to a query will be used to direct which video (and/or audio, graphics, or Web page) 
option will be provided to the viewer. In the present invention, the interactive response to the query 
can occur immediately following the entry of the viewer entry or at some predetermined later time in the 
program using "trigger points/' 500 as explained in detail below. 

11 
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With respect to the video segments, cameras 100 are preferably trained on different segments 
of the sporting event 10. As is common with broadcasts of a football game, for example, cameras 100 
could be located in the endzone, press box, the field and at various other locations throughout the 
stadium. Further, various video options can be created including video replay, slow motion, isolation 
on cheerleaders, particular player or group of players. Instant replays are created by delaying the live 
feed for a certain number of seconds. These video streams are sent to a control studio 5. The control 
studio 5 contains the necessary equipment for packaging the program for dehvery to the viewers. The 
studio 5 contains a video switcher 105 which receives the live signals from the cameras 100 by way of 
various input lines. Further, Unes carrying recorded video streams from one or more VCRs 110, 
computers or CD players feed mto the video switcher 105. The video switcher 105 also receives 
video inputs from the control computer 135. Further, various graphics screens, depicting, for example, 
sports team or player statistics can be designed with the control personal computer 135 and forwarded 
to the digital video switch 105. The producer, via the control PC 135, dkects which video options to 
pass through the video switcher 105. At the output of the video switcher 105, each of the different 
output video streams access a separate encoder 125 and are all GENLOCKED, so that each video 
stream is synchronized with the other video streams. 

After encoding, the video streams are input into a video compressor 125. Preferably, the digital 
compression scheme is MPEG-2. Preferably, 64 Quadrature Amplitude Modulation (QAM) is used as 
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the modulation scheme. In this digital embodiment, four channels of digitally-compressed video content 
would carry about 27 Mbps using 6 MHz of bandwidth. Alternatively, if 256 QAM is employed, 
seven-to-one video compression can be achieved with the MPEG-2 scheme. While MPEG-2 is the 
preferred compression scheme, the signals can be compressed according to any known standard 
including MPEG-1, JPEG, or other DCT coding scheme, wavelets, fractals or other transform or 
waveform based technique. 

The control studio 5 also contains an audio switcher 1 15 which receives live audio signals from 
microphones or recorded audio from tape players 120, CDs, VCRs 1 10, etc.. The control computer 
135 sends commands to the audio switcher 1 15 directing which audio options should pass through the 
switch 115. Further, in the audio switcher 1 15 the various audio signals can be aligned to match the 
various video signals in time. In addition, VCR audio output is received by the audio switcher 115. 
The present invention can accommodate any number of audio signals as output from the audio switcher 
1 15, as directed by the producer. The audio outputs are received by an digital audio 
encoder/compressor 130. The audio signals are then preferably sampled, encoded and compressed in 
the digital audio encoder/compressor 130. The encoding technique can be a waveform coding 
technique such as PCM, ADPCM or DM. Alternatively, the signals can be encoded using synthesizer 
or vocoder techniques such as MUSICAM, Linear Predictive Coding (LPC), Adaptive Predictive 
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Coding (APC), and Sub-band coding. Generally, the transmission rate is about 256 kbps per audio for 
the stereo pair. 

The timing and control for integrating the various multimedia elements is provided by the ACTV 
authoring language, a unique set of interactive data codes to facilitate the interactive process. The data 
codes are stored in memory in the control computer 135 as part of the ACTV programming language. 
The codes comprise commands, or branch codes, for branching between interactive options, timing 
signals for controlling the interactive program, data or text, commands for termination and initiation or 
interactive program viewing, or triggers for executing macros. Preferably, these commands are output 
from the control computer 135 and multiplexed with the video streams in the MPEG-2 compressor 
125, as shown in Figure 1. Interactive options that can be branched to based on the branch codes 
include video segments, audio segments, graphics segments and/or identified Web pages. 

There are several commands in the ACTV authoring language that function to provide the 
interactive sports programming applications. These commands are explained below in Section D, 
which details progranmiing applications. 

B. The Transmission System 

As shown in figure 1, the digital interactive system 1 uses an interactive program delivery 
system with any transmission means including satellite 15, cable 150, wire or television broadcast 175 
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to deliver the interactive program (hereinafter "composite interactive program") from the control studio 
5 for distribution to subscribers in their homes. At the control studio 5, the signals from the digital 
multiplexer 140 are converted to RF and distributed to a microwave 175, cable 150 or satellite 15 
network. Preferably, the digital interactive signal is forwarded from the control studio 5 to a cable 
headend 150, and subsequently, sent to the homes via the cable network. 

The program is preferably the broadcast of a live event. For example, hve sporting events with 
added interactive elements can be broadcast from the control studio 5. Such live interactive elements 
could be different camera angles 100, slow motion video, etc., as discussed above, while also 
incorporating prerecorded interactive segments such as highUghts. Alternatively, the program can be 
produced off-line and stored in a program storage means at the control studio 5. 

In a sateUite broadcast transmission, the digital interactive signals are transmitted to uphnk 
equipment where they may be multiplexed, upconverted, modulated, amplified and transmitted by 
satellite 15 to the receiver site 155 for distribution to the homes. 

At the reception end, the composite digital interactive signals enter a receiver 155 where the 
signals are demultiplexed, downconverted, demodulated and then passed to a cable distribution system 
that directs the signals to the homes. Although a cable distribution system 150 is the preferred 
transmission media to the homes, the digital signals may also be distributed by any conventionally 
known technique including satellite 15 to digital satellite receivers 155 at the home, fiberoptics, low or 
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high power broadcast television 175, telephone lines, cellular networks, and similar technology can be 
used interchangeably with this program delivery system. 

C. The Interactive Digital Box 

The interactive digital box 25 is shown schematically in Figure 2. Preferably, the interactive 
digital box is a specially adapted digital cable box 25. The controller 260 determines what video, 
audio, graphics and/or Web pages to display based upon the interactive commands which it receives. 
Based upon the commands, it plays the appropriate video, audio, graphics or Web page options. The 
graphics can either be created and sent from the control studio 5 or the graphical images can be created 
at the interactive digital box 25 based on instructions preferably in the interactive commands. The 
interactive digital box 25 connects to a television 165 or other display monitor. Further, the interactive 
digital box 25 can be connected to a digital television 195, in which case an RF modulator 245 is not 
necessary. Each downstream transmission reaches the subscriber's house, shown in Figure 2, 
preferably through a tap and drop cable. 

The user interacts with the program through the input device 20. Preferably, the input device 
20 is a typical television remote. The user interface 270 may be an infrared, wireless, or wired receiver 
that receives information from the input device 20. 
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Regardless of the type of input device 20, user inputs can be utilized by the present invention 
immediately, or at a later time, to result in personalized graphics, video and/or audio presentation. For 
example, the present invention utilizes "trigger points," 500 as described belov^, to enable subsequent 
branches among multimedia segments during the show. 

Certain commands are sent from the control studio 5 as part of the digital interactive 
programming to facilitate the collection of user entries. These commands are extracted at the digital 
demultiplexer 210 and sent to the controller 260 which performs the appropriate action based on the 
commands. Some of these commands are explained below. 

a. BEGIN INPUT EXTENDED 

The Begin Input command starts an input period during which the user may press one or more 
buttons to select his or her choice(s). The entry format of this conmand is set forth as follows: 

BEGIN INPUT {{N} {VALID KEYS} {DIFFERENT} {FEEDBACK} 

where: 

N The maximum number of keys that can be pressed. 

KEYS The valid keys that can be pressed. 

DIETERENT Requires each key pressed to be different. 
FEEDBACK The feedback type provided to the viewer. 

b. BEGIN VIDEO CHOICE EXTENDED 

The Begin Video Choice begins an input period for disco mode. During disco mode, the video 
switches dynamically each time the viewer makes a selection. 

BEGIN_VIDEO„ CHOICE MODE [KEYS] {FEEDBACK [AUDIO]} 
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The disco mode allows the viewer to change channels at will, while the OneShot mode allows 
only one change of channel. 

MODE DISCO/ONESHOT 

KEYS The valid keys that can be pressed. 

FEEDBACK The feedback supplied to the user for the key(s) which are pressed. 

The mode Disco allows the viewer to change channels at will, while the OneShot mode allows 
only one change of channel. 

c. BEGIN A UDIO CHOICE EXTENDED 

The Begin Audio Choice begms an input period for disco mode. 
BEGIN_AUDIO_ CHOICE MODE [KEYS] {FEEDBACK [AUDIO]} 

MODE DISCO/ONESHOT 

KEYS The valid keys that can be pressed. 

FEEDBACK The feedback supplied to the user for the key(s) which are pressed. 

d. MAP 

The Map command is used to map video or audio tracks to keys, for use in connection with the 
Begin Audio Choice and Begin Video Choice commands. If this command is omitted, Key 1 
will map to Track or Channel 1, Key 2 to Track or Channel 2 etc. This command allows 
mapping any key to any channel. 

MAP KEYS [TRACKS [T1,...TN/T1-TN] CHANNELS [C1,...CN/C1-CN]] 

The map statement maps audio tracks or video channels to keys, to enable audio or video 
choice commands to effect changes to tracks other than the default tracks, which are that key 1 
maps to track 1, key 2 to track 2, etc.. All the choice statements after a map statement will 
cause the tracks, video tracks or channels to be changed to those specified in the map 
statement. The acceptable numbers for the map conmiand are from 1 to 8, for keys, video 
channels and audio channels. 
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Other commands include those which allow for the following applications: (1) viewer profiling, 
to enable the set top box 25 to "remember" viewer preferences; (2) uploading viewer responses to a 
central location; (3) downloading of text and graphics, for display using the graphics chip of the set top 
box 25; (4) the ability of the viewer to prepare his own video, based upon his selections of camera 
shots 100 and audio, which can be stored and replayed for the viewer. 

The interactive digital box 25 of the present invention enables seamless flicker-free transparent 
switching between the digital video signals. "Seamless" means that the switch from one video signal to 
another is user imperceptible. Because the video signals are running off the same clock, the interactive 
digital box 25 is capable of providing a seamless digital switch from one video signal to another signal. 
The program clock reference necessary for the box to make this seamless switch is preferably 
embedded in the signal header. 

As shown in Figure 2, a CPU 260 is connected to an RF demodulator 200 and digital 
demultiplexer 210. The CPU 260 directs demodulation and demultiplexing of the proper channel and 
data stream to obtain the correct video signal. Seamless switching can occur with MPEG-2 
compressed signals since there are points within the frame wherein seamless switching can occur. 
Preferably, switches occur at an "I" frame, assuming the use of MPEG-2 compression. The selected 
video signal is determined either by examination of the user's input from user interface 270 and/or any 
other information or criteria (such as personal profile information) stored in RAM/ROM 265. For 
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example, the RAM/ROM 265 could store commands provided within the video signals as discussed in 
U.S. patent no. 4,602,279, and incorporated herein by reference. 

The RF demodulator 200 demodulates data from the broadcast channel directed by the 
controller 260. After the data stream is demodulated, it passes through a forward error correction 
circuit 205 into a digital demultiplexer 210. The demultiplexer 210 is controlled by the controller 260 
to provide a specific video signal out of a number of video signals which may be located within the data 
stream on the demodulated broadcast channel. The demultiplexed video signal is then decompressed 
and decoded by decompressor/decoder 215. The video signal is synchronized by a sync add circuit 
220 and a sync generator 225. The video signal is then buffered by a video frame buffer 230. The 
buffered video signal is modulated by a modulator 245 into a NTSC compatible signal. Such a 
modulator is not necessary if the selected signal is sent to a digital television 195. 

By using a video frame buffer 230 and delaying the viewing of a given signal, enough time is 
allowed for the decompressor/decoder 215 to lock onto, decompress, convert to analog, and wait for 
the resultant vertical interval of a second video signal. For example, assume video signal A is currently 
being processed and transferred through the circuit shown in Figure 2 and displayed. Based upon a 
user selection, the controller 260 directs the digital demultiplexer 210 and RF demodulator 200 to 
switch to another video signal, video signal B. To accomplish this, the analog video from the first digital 
video signal, video signal A, complete with video sync, is fed into video frame buffer 230. This buffer 
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230 can hold the full video picture for "n" number of frames after which the signal is output to the 
display. In effect, a delayed video signal A is viewed "n" number of frames after the signal has been 
received. When the user selects a different video path by means of pressing a button on a keypad or 
entry by other means, the controller 260 instructs the digital demultiplexer 210 to stop decoding signal 
A and lock onto signal B to begin decoding signal B instead of signal A. 

While this is happening, even though the decompressor/decoder 215 is no longer 
decompressing video signal A, the display is still showing video signal A because it is being read from 
the buffer 230. As soon as decompressing and decoding occurs, the controller 260 looks for the next 
vertical blanking interval (VBI) and instructs the video frame buffer 230 to switch to its input, rather 
than its buffered output at the occurrence of the VBI. 

Since the RF demodulator 200, forward error corrector 205, digital demultiplexer 210, and 
decompressor/decoder 215 require a certain time period to decompress and decode the video signal B 
frame from its data stream, the size of the buffer 230 has to be large enough so that this processing can 
take place without interruption during the switching of the video signals. If desired, the system may 
continue to use the buffer in anticipation of a future switch. By using the controller 260 to manipulate 
the fill and empty rate of the buffer 230, the buffer 230 may be rapidly filled with video signal B frames 
and then after a period of time will be reset and ready to make another switch to another video in the 
same manner. The buffer 230 may also be reset by skipping frames or providing a delay between 
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sequential frame outputs for a short time in order to fill the buffer 230. If a delay is used to maintain 
video signal or frame output while the buffer 230 is being filled, a slight distortion may occur for a brief 
amount of time. 

Because a first video signal is always displayed as the output of the buffer 230 after the delay, 
the buffered video masks the acquisition and decoding of a second video signal. As long as the buffer 
230 is large enough to keep the first video running while the second video is being decompressed and 
decoded, a seamless switch will occur. 

While the digital interactive box 25 of Figure 2 provides video interactivity, audio and/or 
graphics interactivity is also provided. For example, if, based on the viewer profile or viewer response 
to query, it is determined that the viewer's primary language is Spanish, then that viewer could obtain 
Spanish commentary to the football, soccer, etc. game. Alternatively, if a viewer has a favorite athlete, 
the audio can switch to an interview with the athlete during a segment of the broadcast. Multiple digital 
audio options forming a set of suitable responses to an interrogatory message can be sent as part of the 
composite digital signal. As set forth in U.S. patent no. 5,585,858, herein incorporated by reference, 
there are a number of different ways to effectively forward the necessary audio options for a given live 
event to the digital interactive box 25. With the present invention, it makes no difference how the audio 
options reach the digital interactive box 25, as long as they are available for selection and play at the 
appropriate times. 
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In Figure 2, the digital demultiplexer 210 extracts the digital audio signal(s) and forwards them 
to the audio switch 250. Additional audio options are available from the digital audio memory 255. At 
certain times during the program, the data codes will identify the selection of a particular audio option 
corresponding to previous user inputs. The controller 260 calls the appropriate audio options from 
internal memory 255 or directs the audio switch 250 to select a predetermined audio segment received 
as part of the received digital signal for passage to the RF modulator 245 for play to the subscriber. At 
the end of the audio segment time period as indicated by the data codes, the controller 260 instructs the 
audio switch 250 to again pick up standard audio. 

The digital demultiplexer 210 sends the extracted graphics data or ACTV data codes to the 
controller 260. The controller 260 interprets the extracted data as either control data, including 
instructions for switching between video signals, audio signals, or graphics data for on-screen display. 
If the data is on-screen display data, the data is preferably prefixed by a command designating the data 
as on-screen display data, as opposed to control data. Further, the controller 260 also examines the 
control data for the occurrence of a header code designating the onset of a trigger point 500 in the 
program, explained below. 

Figure 3 shows an alternate, dual tuner embodiment for seamless switching between separate 
video signals. This embodiment presumes that two 6 MHz channels are used, each of which comprises 
compressed digital video and audio streams. In this embodiment, the microprocessor 260 controls the 
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selection of the RF channel that is demodulated by RF demodulators 200 A, 200B. The demodulated 
data streams enter the forward error correctors 205 A, 205B. At the output of the forward error 
correctors 205 A, 205B the data streams are transmitted to the input of the digital demultiplexers 21 OA, 
210B. 

As with the RF demodulators 200A, 200B, the digital demultiplexers 210A, 210B are 
controlled by the microprocessor 260. This configuration allows the microprocessor 260 to 
independently select two different individual time-multiplexed video signals on different channels and 
data streams. If all the video signals of an interactive program were contained on a single channel or 
data stream, it would only be necessary to have a single RF demodulator 200, forward error corrector 
205, and digital demultiplexer 210 serially connected and feeding into the two digital video buffers 
230A, 230B. 

Two data streams are provided from the digital demultiplexers 210A, 210B. One data stream 
carries video information pertaining to the video signal the user is currently viewing. The second data 
stream carries the video signal selected based on the user's previous and/or current interactive 
selections from the user interface 270, as determined by the microprocessor 260. 

The digital information on each of the two streams is buffered in digital video buffers 230A, 
230B. The buffered signals are then decompressed and converted into analog signals by 
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decompressors/decoders 215A, 215B which include digital to analog converters. The decompressors 
215A, 215B are preferably MPEG-2 decoders. 

A local sync generator 225 is connected to sync add 220A, 220B and frame sync 380A, 380B 
circuits. Because both streams are synchronized based on signals from the same local sync generator 
225, each stream becomes synchronized to the other. In particular, the signals on each stream are 
frame synchronized. 

A vertical blanking interval (VBI) switch 335 is connected to the microprocessor 260 so that 
the input may be switched during the vertical blanking interval of the current stream, resulting in a 
seamless switch to the viewer. 

The embodiment of Figure 3 operates as follows. Based on user responses and control codes, 
it is assumed that the microprocessor 260 determines that a switch from video signal A to video signal 
C should be performed. The RF demodulator 200 A and digital demultiplexer 210A are processing the 
currently viewed video signal, video signal A, which is progressing through the upper branch 
components. A command is issued from the microprocessor 260 to the RF demodulator 200A, 200B 
commanding a switch to the channel and data stream on which video signal C is located. The 
microprocessor 260 also instructs the digital demultiplexer 21 OB to provide video signal C from the 
received data stream to digital video buffer 230B. 
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At this point, the upper RF demodulator 200A and digital demultiplexer 210A are still 
independently receiving and processing video signal A, which continues through the upper branch of the 
circuit. 

At a certain point, the digital decompressor/decoder 215B in the lower branch will begin filling 
up with video signal C frames. After video signal C is decompressed and decoded, it is converted into 
analog. A local sync generator 225 inserts both local sync and frame sync to video signal C via sync 
add circuit 220B and frame sync circuit 380B in order to synchronize it with the currently displayed 
video signal A, which is still being provided from the upper digital video buffer 230A. At the 
appropriate switch point, triggered by programming codes supplied with each video signal A and C, the 
microprocessor 260 directs the VBI switch 335 to switch in the vertical blanking interval from video A 
to video C, at which time video C will then seamlessly appear on the computer screen. 

Digital video buffers 230A, 230B may be used in the circuit of Figure 3, but are optional. 
However, in an alternative embodiment the buffers would be required to provide a seamless switch if 
the Figure 3 circuit was modified to incorporate a single RF demodulator 200, single forward error 
corrector 205, and single digital demultiplexer 210, each with a single input and single output. In this 
alternative embodiment, the circuit cannot independently receive and demultiplex two data streams on 
different frequency channels. One buffer 230A is used to store previously received video signals, while 
the other buffer 230B quickly passes through the selected video signals. 
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Based on the same assumptions above, video signal A is progressing through the upper branch 
of the circuit and it is desired to switch to video signal C. However, in this alternative embodiment, the 
digital video buffer 230A is providing maximum buffering to video signal A. 

Because it is desired to switch to video signal C, the microprocessor 260 directs the alternative 
circuit (containing a single RF receiver 200, single forward error corrector 205 and single digital 
demultiplexer 210 connected in serial), to receive and demultiplex the data stream on which video signal 
C is located, which may be different than that of video signal A. When video signal C is demultiplexed, 
the microprocessor 260 directs the digital video buffer 230 to provide minimum buffering of video 
signal C so that decompressor/decoder 215 may quickly decompress and decode the digital signals. 
After decompression and decoding, video signal C is synchronized with video signal A. At this time, 
video signal A is read for display from digital video buffer 230A. The upper digital video buffer 230A 
must be large enough to provide video frames for output during the time it takes the RF demodulator 
200 and digital demultiplexer 210 to switch to video signal C and the time required for decompression, 
decoding, and synchronization of video signal C. 

When video signal C is synchronized with video signal A, the microprocessor 260 directs VBI 
switch 335 to switch from video signal A to video signal C in the vertical blanking interval of video 
signal A, thereby providing a seamless and flicker-free switch. 
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At this time, digital video buffer 230 will begin to utilize maximum buffering by altering its 
fill/empty rate as described above with respect to the Figure 3 embodiment. When adequate buffering 
is achieved, a switch to another video signal may be performed in the same manner as described above. 

Another preferred embodiment is shown in Figure 4. This embodiment also includes an RF 
demodulator 200, a forward error corrector 205, and a digital demultiplexer 210. However, the 
circuitry differs along the rest of the chain to the television set or monitor. In this embodiment, a 
memory 475 is incorporated and connected to the output of the demultiplexer for storing the 
compressed composite digital video signal. The decompressor/decoder 215 is inserted at the output of 
the compressed memory. The decompressor/decoder 215 decompresses the digital signal, converts 
the signal to analog and forwards the analog signal to the RF encode 245 for transmission to the 
monitor. Once the composite compressed digital video signal is fed into the compressed memory 475, 
the microprocessor 260 directs a pointer to be placed somewhere along the compressed digital video 
signal. Based on the placement of the pointer, different frames and different segments of the composite 
digital video signal will be read from memory 475 for decompression and decoding. 

The different video signals are distinguished from one another because they are labeled, 
preferably by headers. Assuming that video signal A has been selected for play on the monitor, the 
compressed digital memory 475 fills up with A frames. Assuming a switch to video signal C is desired, 
the microprocessor 260 directs the RF demodulator 200 and digital demultiplexer 210 to begin filling 
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the compressed memory 475 with video C frames. The decoder 215 pointer begins to move down. 
As soon as a sufficient number of C frames have entered the compressed memory 475, the pointer will 
then jump to the beginning of the C frames. The C frames are then output into the 
decompressor/decoder 215 where the digital frames are converted into an analog signal. 

The digital video is multiplexed in a series of easily identifiable packets. These packets may 
contain full compressed frames of video (I frames) or may include only the differences between full 
frames (B frames or P frames). 

To be able to reconstruct the full video images, the decompressor/decoder 215 needs to have a 
minimum number of I, P and B frames. The decoder 215 needs only one I frame to decode an image. 
Conversely, two prior Anchor frames ("I's" and "P's") are necessary to decode B frames. In order to 
decode P frames, the decoder 215 only needs one Prior Anchor frame. When the microprocessor 
instructs the digital demultiplexer 210 to start sending packets from a different data stream there is no 
way to be certain that the next packet will be an I packet needed for decoding the second video 
stream. To avoid a breakup of the video images, which would occur if the decompressor/decoder 215 
suddenly started receiving packets unrelated to the stream it was decoding, the microprocessor 260 
starts to fill up the memory 475 with video signal C packets until it is determined that a full sequence of 
I, B and P frames are available. The decoder 215 should receive the last bit of the last B frame in a 
given, GOP (Group of Pictures) before the switch, in order to prevent glitches when decoding. 
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Furthermore, the last B frame of the GOP must only be backward predicted, not forward predicted or 
bidirectional predicted. As soon as the valid sequence is in memory 475 the microprocessor 260 
moves the memory read pointer to the start of a valid sequence of C video signal packets so that the 
decompressor/decoder 215 can successfully decode the C signals. This results in a seamless switch 
from video signal A to video signal C. 

This embodiment requires a data channel for enabling a synchronous switch between a first 
video stream and a second video stream. This data channel comprises the ACT V codes which link 
together the different program elements and information segments on the different video signals. In 
addition, the data channel also comprises synchronization pulses and a time code to signify to the 
pointer the proper time to skip from a memory location representing one video signal to a memory 
location representing another video signal in order to enable a seamless switch. 

The microprocessor 260 reads the data signal from the digital demultiplexer 210 and 
communicates pertinent data to the sync add circuit 220, which is connected to sync generator 225. 
The microprocessor 260 is then able to synchronously communicate with the memory 475. 

The time code sent will identify the timing for one picture, as well as for multiple pictures, and 
will lock the different pictures together. This is done through the use of similar clocks at both the 
transmission end and the receiver. A time code is used in order to keep the two clocks at both the 
transmission and receive end synchronously connected to one another. Once the clocks at both ends 
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are working synchronously, each of the multiplexed video streams must be synchronized to the clocks. 
In order to synchronize the multiplexed video stream to the clocks, each of the individual channels must 
be referenced to a conunon reference point and must be identified. 

In the preferred embodiment, a packet header would be incorporated into the transport layer of 
the MPEG signal to identify the various channels. The packet header will also include information as to 
where to insert the vertical blanking interval. In MPEG, the vertical blanking interval is not transmitted 
from the headend. Therefore, the vertical blanking interval must be generated locally. The packet 
header eye will identify at what time the vertical blanking interval is in existence in order to effectuate a 
seamless switch between analog pictures. 

In summary, the combination of clock and the information embedded in either the transport 
layer of MPEG or in a separate packet on a separate data channel effectuates the linking between each 
video signal and a corresponding time point. The data channel also includes information designating 
when all the various video signals will be in synchronism with one another. It is at these points that the 
microprocessor 260 may direct the pointer to skip from one location to another location, at a time (such 
as during the VBI) when a seamless switch will result. 

D. Trigger Points 
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Interactivity is further enhanced in the digital interactive embodiments through the application of 
trigger points 500 scattered at various predetermined times throughout the program, a timeline 
representation of which is shown in Figure 5. The trigger points 500 correspond to times when 
interactive events are scheduled to take place during the live sporting event 10. These interactive 
events could be the selection and playing of video, audio segments, the display of graphics or display of 
Web pages accessed from Internet Web sites 170. For example, when a viewer's favorite baseball 
player is at bat, graphics showing past or current performance statistics of the player can be overlaid on 
the screen while excerpts from an interview with the player can be played for the viewer. While the 
choice of particular video, audio or graphics is still dependent on viewer selections, the viewer 
selections in response to displayed graphical interrogatory messages are preferably made during a 
period at the onset of the program or when a viewer first tunes into the program. Alternatively, 
interrogatories are not necessary if the switches are based on the viewer profile stored in memory 265. 
These viewer selections are then utihzed as inputs to macros called up at later times during the program 
by the controller 260 upon the occurrence of the trigger points 500, identified to the interactive 
computer by unique codes embedded in the video signal. 

The trigger points 500 correspond to the times when the conventional program content can be 
altered and personalized for the viewers. The progranomer can place the trigger points 500 at any time 
throughout the program. Since the trigger points 500 are unknown to the subscriber, the subscriber 
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does not know when they will receive a personalized message. In other words, an interactive response 
can either immediately follow a corresponding user selection made to an interrogatory message or 
occur at a later time corresponding to a trigger point 500, or any combination of the two. Of course, 
timing of the interactive events should correspond to suitable times in the program where branching to 
interactive elements is sensible and does not clash with the program content of the conventional video 
still displayed on the television 165 or other display monitor. 

At the onset of a trigger point 500, the controller 260 will select one of several possible audio 
(or video or graphic display) responses for presentation to the subscriber. As mentioned above and 
shown in figure 5, some of the responses may comprise a branch to either a video segment, graphics 
and/or audio segments. 

In combination with the use of trigger points 500, the present invention allows for the viewer to 
select certain options at the onset of the program to suit the viewers' preferences. For example, if the 
program broadcast is a live sports event 10, at an early trigger point 500, the viewer could be queried 
as to whether the viewer would prefer to receive audio in English, Spanish, French, or perhaps hear the 
local announcer instead of the network announcer. Upon the viewer selection, the CPU 260 directs a 
branch to the appropriate interactive segment. 

Each trigger point 500 is identified preferably through the broadcast of ACTV codes sent as 
part of the composite interactive program signal. The codes preferably include, at a minimum, the 
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following information: (1) header identifying the occurrence of a trigger point 500; (2) function ID 
(e.g., selection of audio or graphics responses, etc.); and (3) corresponding interrogatory message(s) or 
particular viewer characteristic or habit based on viewer profile. The first bit sequence simply identifies 
to the controller that a trigger point 500 is about to occur. The function ID designates the macro or 
other set of executable instructions for the controller 260 to read and interpret to obtain the desired 
result, e.g., a selected video and/or audio response. 

Upon extraction of the codes by the data decoder, the controller 260 reads and interprets the 
codes and calls from memory 265 a particular user selection(s) designated by the trigger point 500 
codes. The user selections correspond to subscriber answers to a series of interrogatory messages 
preferably presented at the beginning of the program. After obtaining the appropriate user selection(s), 
the controller 260 reads and performs the executable instructions using the user selection(s) as input(s) 
in the macro algorithm. The result of the algorithm is either a selected video stream, audio and/or 
selected graphics response. The video/audio response can be called from memory 265 if it is 
prestored, called from external data storage, or the controller 260 can command the switch to branch 
to the particular video audio stream if the response is broadcast concurrently with the trigger point 500. 
After the selected video/audio response is played to the subscriber, the switch branches back to the 
standard program, shown at time t^ in Figure 5. 
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As mentioned above, a series of interrogatory messages are preferably presented when the 
subscriber begins watching the interactive program. These interrogatory messages can be presented in 
any one of three ways. First, the interrogatory messages can be presented as graphics displays overlaid 
by the interactive computer workstation onto a video signal, wherein the graphics data is sent in the 
vertical blanking interval of the composite interactive signal, or alternatively stored on the hard disk or 
external storage. Second, the interrogatory messages are presented as graphics displays as discussed 
above, except the graphics data comes from local storage, external data storage (e.g., CD ROM, 
cartridge, etc.), or a combination of data in the VBI and data called from either local or external data 
storage. Third, graphics data can be presented in the form of user templates stored at the interactive 
computer workstation. 

User selections corresponding to answers to the n successive interrogatory messages are 
received by the remote interface 270 at the beginning of the show, stored in memory 265 and used 
throughout the show at the appropriate trigger points 500 to subtlety change program content as the 
show progresses. Preferably, each interrogatory has a set of possible answers. Next to each possible 
answer will be some identifier corresponding to a label on a key on the user interface. The subscriber 
depresses the key corresponding to their answer selection. This selection is decoded by the remote 
interface 270 and controller 260, stored in memory 265, preferably RAM, and used later as required 
by an algorithm designated at a trigger point 500. 
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E. Internet 

In addition to the central studio serving as a source of interactive option responses, the Internet 
can be used as a source of personalized information for interactive responses. 

As discussed above, the video programming is preferably created at a centralized location, i.e., 
the control studio 5 as shown in Figure 1, for distribution to subscribers in their homes. Referring to 
Figures 1 and 6, in a preferred method, the operator at the control studio 5 must designate certain Web 
pages to correspond to one of the program options, such as audio and/or graphics options, using 
control PC 135 and URL encoder 600. Instead of encoding the actual content of the options at the 
control studio 5, as with the audio, Web address identifiers, i.e.. Uniform Resource Locators (URLs) 
are encoded and sent as part of the data codes from the control PC 135 to the digital multiplexer 140. 
After multiplexing, the URLs are sent as part of the program signal 615, as described above. 
Preferably, the URLs, like the various audio and graphics options, have associated time stamps which 
indicate to the remote digital set top boxes 25 when, during the video program, to display the particular 
Web pages addressed by the URLs, the selection and display of which is preferably made as a function 
of viewer responses or viewer profile. 

Preferably, each digital set top box 25 has an Internet connection 160 created concurrently 
with the cable connection. The Internet connection 160 can be via high-speed line, RF, conventional 
modem. The digital set top box 25 has Intemet access 160 via any of the current ASCII software 
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mechanisms. In a preferred embodiment, in the interactive digital set top box 25, the digital 
demultiplexer 210 extracts the URLs along with the other data codes. In an alternative embodiment, a 
local URL decoder 605 at the user site extracts the URLs. 

In a preferred embodiment, a JAVA enabled browser as well as specialized software for 
performing part of the method of the present invention are installed on the interactive digital set top box 
25. The JAVA enabled browser allows the interactive digital set top box 25 to retrieve the Web pages 
and is preferred software, since it is platform independent, and thus, enables efficient and flexible 
transfer of programs, images, etc., over the Internet. The specialized software acts as an interface 
between the video programming and the Intemet functions of the present invention. The processor and 
software interprets these URLs and directs the JAVA enabled browser to retrieve the particular 
relevant Web pages, and synchronizes the retrieved Web pages to the video content for display on the 
television monitor 165 at the appropriate times. 

In the present invention, the viewer also has the capability to link to a channel website at will. 
For example, if a viewer is interested in purchasing a product described in an advertisement, by merely 
clicking on a button on their remote 20, the producer's Website could be accessed by Intemet 
connection 160 and displayed to the viewer. The viewer could then either obtain more information 
about the product or order the product, if desired. As described above, this application is possible by 
sending the URL associated with the producer's Website to the digital cable boxes 25 as part of the 
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interactive program. Upon selection by the viewer, the web browser, located either in the digital set- 
top box 25 or externally in a connected PC 610, can retrieve the Web pages. The specialized software 
then synchronizes the Web pages for video display. 

F. Memory 

The interactive digital set top box 25 of the present invention also has the advantage of 
remembering subscriber responses and using these responses in choosing a video/audio response, 
and/or graphics interrogatory message, to present to the student. Memory branching is a technique of 
the present invention where the algorithm assembles video/audio responses and graphics interrogatory 
messages according to the current and previous user inputs. Memory branching is accomplished by 
linking video/audio streams and/or successive graphics interrogatory messages together in a logical 
relationship. In this scheme, the interactive digital set top box 25 contains logic (preferably, in the 
software algorithm) and memory 265 to store previous subscriber selections and to process these 
previous responses in the algorithm to control future video/audio stream selection, as well as future 
graphics message selection. 
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G. Digital Viewer Profiles 

In a preferred embodiment, the interactive digital cable box 25 can have a "viewer profile" 
stored in its memory 265. Alternatively, the accumulated profile of viewer characteristics and/or habits 
can be stored at the control studio 5 or cable headend. If the profile statistics are accumulated at some 
central location, they can also be sent to each viewer's home. On the other hand, if accumulated in 
memory 265 at each of the set top terminals 25, the data could be sent to the central location for 
storage and dissemination by way of the digital back channel Decisions regarding personalized 
advertising and viewing can then be made for a viewer or class of viewers based on the accumulated 
parameters. 

The "viewer profile" preferably contains characteristics of the particular viewer at that 
subscriber location, such as sex, hobbies, interests, etc. This viewer profile is created by having the 
viewer respond to a series of questions. Alternatively, the viewer profiles could be created at a control 
studio 5 and sent to the interactive digital cable box 25. This information is then used by the cable box 
software to create a compendium of the viewer's interests and preferences - i.e., a user profile. The 
stored user profile would be used in place of the question/answer format, and thus, dictate the branches 
to interactive segments of interest to the viewer. 

Alternatively, the interactive cable box 25 can be programmed to create a user profile of each 
viewer based on the selections made during one of the interactive programs. If the profile shows that a 



39 



PATENT APPLICATION 
Attorney Docket No. 5038.25 
Express Mail Label No, EF084879952US 

particular viewer does not enjoy violence, the system can automatically branch to another video signal 
at the commencement of a fight in a sports program. Furthermore, such a user profile could be 
modified or enriched over time based on selections made during future interactive programs. For 
example, the 'memory' technique described above can be used to modify the user profile based on 
user response over time. 

Event data is collected from the viewer when the following command is received and processed 
by the controller 260 in the interactive digital cable box 25: 

VIEWER PROFILE 

This command is used to enable events which can be utihzed for profiling during a show. The 
events indicated in this command are those profiling events which are possible during the show. 
The enabled events may be either selected by the viewer during the show, or may be 
automatically enabled based upon viewer selections. 

VIEWER„PROFILE EVENT1,..,EVENT24 

Once the profile is created, the progranmiing choices or interactive responses can be triggered 
based on the content of the viewer profile itself. For example, if the viewer profile suggests that the 
viewer is particularly interested in sports cars, a sports car commercial could be played for the viewer 
at a predetermined point in the program. As another application, if a viewer's profile indicates that the 
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viewer is interested in cooking, whenever the viewer watches such a program, the user profile would 
trigger the interactive program to download recipes and either display such recipes on the screen or 
send the recipes to an attached printer. 

Viewer profile information can then be collected at the control studio 5 through polling of the 
viewer cable boxes 25 for viewer selection data. Special polling software is loaded into the digital set 
top box 25 for performing the polling functions. Alternatively, the invention allows for the digital set top 
boxes 25 to send back data on command from the control studio 5 or periodically. The command to 
initiate an upload of viewer profile data is as follows: 



UPLOAD EXTENDED 

This conmiand is used to initiate the uploading of data to the central site. 

UPLOAD_EXTENDED UPLOAD_IDENTIFIER VARIABLE_NAME 
[PHONE^NUMBER] 

UPLOAD__IDENTIFIER numeric constant identifying upload. 
VARIABLE_NAME name of variable to be uploaded 

PHONE__NUMBER string, that consists of numbers only, it can be 

omitted, if not needed. 



Regardless of whether a polling or periodic scheme is used, the statistics and other user profile 
information is preferably sent back to the control studio 5 by use of the back-channel. 
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H. Applications 

The embodiments, described above, allow for several possible applications. For example, in a 
live sports event 10, one channel could carry the standard video channel, with other channels carrying 
different camera angles 100 and/or close-ups of particular players. Other potential video options 
include instant replay, highlights, player statistics via graphic overlays, etc. Graphics presenting 
statistical information on the players can be constantly updated using the Chyron system. 

The provision of trigger points 500, explained above, can provide for seamless integration of 
such video options during the sporting event, based on either the viewer responses to interrogatories at 
the beginning of the program and/or on the digital viewer profile. 

Further, the viewer can become the director with the present invention. For example, the 
viewer can choose which camera angles 100 to emphasize. In a broadcast of golf, the viewer can 
direct whether they desire to follow a particular player from hole-to-hole, focus on one particularly 
difficult hole. In this manner, the viewer can customize the sporting broadcast to meet his own 
interests. 

Further, the viewer can act as a director to create their own video. During a live concert 10, 
for example, the viewer selects various camera angles 100 at different times. These selections along 
with a time stamp, indicating the program time when each selection was made, are stored in memory 
265. When the program is played back a second time, the processor 260 will automatically direct 
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branching between the video channels according to the stored selections at the time stamp. In this 
manner, a music video can be created by the viewer. 

The viewing experience can be further enlightening for the viewer by implementing games and 
contests during the live sporting event 10. For example, graphics overlays can be developed that query 
the viewer during the game. During a football broadcast, for example, viewers can be queried with 
such interrogatories as the following: 

What will be the next play? (RUN/PASS/KICK); 

Will the offense get the first down?; 

Will they score on this possession?; 

Pick the half time score; 

Who will win? 

Each viewer's responses can be sent back to the control studio 5 for tabulation of scores. 
Preferably, the responses are packaged at the digital cable box 25 and transmitted to the control studio 
5 via the digital backchannel upon the UPLOAD EXTENDED command. Alternatively, tabulation 
of scores can take place at the digital cable box 25 through the utilization of certain software in memory 
265. Each correct answer can correspond to a certain number of points. At the end of the game, the 
interactive program preferably presents a graphic showing the viewer point total. If desired, advertisers 
could present special gift certificates for excellent performance in such games. The provision of such 
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certificates would occur by displaying a certain code that a viewer can take to a store to receive the 
gift. In this manner, viewer interests in sports events can be enhanced. 

Further, the viewer has the option with the present invention to block out viewing of certain 
events. For example, if the viewer is adverse to violence during a sporting event or other type program, 
the system can block out such options from the viewer with the following data commands: 

EVENT 

This conomand is used to indicate occurrence of a certain event (e.g., a fight breaking out during 

a football game). 

EVENT EVENT_NUMBER 

EVENT_NUMBER is a numeric constant. 

ON EVENT EXECUTE MACRO 
This command is used to implement certain actions (such as an automatic branch to another 
video channel in order to block out violent event, for example), as soon as the event arrives. 
ON_EVENT EVENT_NUMBER MACRO_NUMBER 
EVENT_NUMBER is a numeric constant. 
MACRO NUMBER is a numeric constant. 
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1. Two-Way Configuration 

The live progranuning system of the present invention nnay be operated in a two way 
configuration, as illustrated in Figure 7. In this mode, the various video signals are processed as 
previously described, being digitized and compressed at the control studio 5. The signals are then sent 
to a central switching station, or headend 30. 

In this embodiment, the switching between the various live digital signals is accomplished at the 
headend 30 rather than at the receiver. On the receive end, each digital set-top box 760 relays viewer 
selections back to the remotely located switching station 30, Preferably, the viewer selections are 
relayed by way of the digital back channel 770. However, the viewer selections may be relayed to the 
switching station 30 by any conventional means, such as two-way cable television, telephone or 
microwave transmission. The switching station 30 receives the viewer selection and routes the desired 
signal to a transmitter 750 which conventionally transmits the desired video down the appropriate digital 
cable channel for the particular viewer. 

At the central switching station 30, a demultiplexer 710 demultiplexes the compressed signals 
and places each on a separate bus channel 725. A number of remote control interactive switches 730, 
732, 734, 736 are connected to the video signal bus 725. Based on the viewer selections, an algorithm 
stored in memory 265 and under processor 260 control at the central switching station 30, a digital 
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seamless switch is made and the selected video, audio and/or graphics are forwarded to the viewer 
home for display. 

Such a two-way embodiment could be implemented in a video dial tone or video server system. 
In such a system, only a single video channel 755 is necessary for each home. Once the viewer 
selection is received at the server site at the cable headend 30, a switch is made to the appropriate 
video stream and this stream is sent on the single channel 755 to the home. 

Alternatively, it may be desirable to transmit an interactive sporting event over a single 
telephone line. When the viewer enters a selection on their remote 20, a signal is sent by way of the 
telephone line to the central switching station 30 which routes the desired signal of the interactive 
program over the user's telephone line so that a single link handles both the interactive choice being 
made at the receiver and the transmission of that choice from the headend 30 where the actual 
switching takes place in response to the interactive selection made at the receiver. 

The two-way link between the viewer and switching station 30 may be used for other purposes. 
For example, demographic data may be transferred from the viewer to the broadcast network for 
commercial purposes, such as targeted advertising, billing, or other commercial or non-commercial 
purposes. 

While the present invention has been described primarily with respect to live events, and in 
particular sporting events, it has equal potential for enhancing content in other program categories. 
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A viewer can become their own director of a murder mystery or other drama. By entering responses 
to displayed questions at the initiation of or during the show, the program will branch to alternative 
video/audio segments as a result of the user selections. In this manner, different viewers with different 
selections may end up with a different murderer at the conclusion of the broadcast. 

Using the foregoing embodiments, methods and processes, the interactive multimedia computer 
maximizes personalized attention and interactivity to subscribers in their homes in real time. Although 
the present invention has been described in detail with respect to certain embodiments and examples, 
variations and modifications exist which are within the scope of the present invention as defined in the 
following claims. 



47 



PATENT APPLICATION 
Attorney Docket No. 5038.25 
Express Mail Label No. EF084879952US 

CLAIMS 

We claim: 

1 . A system for providing live interactive digital programming, comprising: 

a means for receiving video signals from a plurality of video cameras, one or more of the 

cameras relaying a different predetermined view of a live event; 

a means for producing one or more audio signals corresponding to the live event; 
a means for generating one or more graphics signals; 

at least one digital compression device, connected to the receiving and producing means, for 
digitally compressing the video, graphics and audio signals; 

a means for processing, connected to the compression device, wherein the processing means 
creates a set of data commands which link together the various audio, graphics and video signals, the 
data conomands including branching conmiands; 

a digital multiplexer, connected to the digital compression device, for multiplexing the video, 
graphics and audio signals, and the data codes into a combined digital program stream; and 

a means for transmitting the combined digital program stream. 
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2. A method for providing live interactive digital programming, comprising the steps of: 

obtaining video signals from a plurality of video cameras, one or more of the cameras relaying a 
different view of a live event; 

producing one or more audio signals corresponding to the live event; 

creating one or more graphics signals; 

receiving the video and audio signals in a control studio; 

digitally compressing the video, graphic and audio signals; 

producing a set of data codes corresponding to the programming, the data codes including 
branching commands; 

digitally multiplexing the video, graphics and audio signals, and the data codes into a combined 
digital program stream; 

transmitting the combined digital program stream; 

receiving the combined digital program stream at a receive site; 

re-transmitting the combined digital program stream on a digital cable television distribution 

system; 

receiving the combined digital program stream at one or more viewer sites; 
gathering viewer specific information; 
processing the data commands; 
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digitally demultiplexing the video and audio signals resulting in a first video and audio signal, the 
first output video and first audio signal selected based on the data commands and gathered viewer 
specific information; 

instructing the digital demultiplexer to commence demultiplexing a second video and second 
audio signal, the second video signal and second audio signal selected based on the data commands 
and gathered viewer specific information; 

seamlessly switching from the first to the second video signal; and 

displaying the second video signal on a screen. 

3. The method of claim 2, further comprising the steps of: 

creating a viewer profile with the gathered viewer specific information; 

wherein selecting the video and audio signals are based in part on the viewer profile. 

4, The method of claim 2, wherein the step of gathering viewer specific information comprises the 
steps of: 

displaying at least one interrogatory to the viewer, the content of the interrogatory involving 
program options; 

collecting entries from the viewer in response to the interrogatories; and 
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wherein the selection of video or audio signals is based in part on the collected viewer entries. 

5. A system for providing live interactive digital programming, comprising: 

a means for receiving video signals some of which are from a plurality of video cameras, one or 
more of the cameras relaying a different predetermined view of a live event; 

a means for producing one or more audio signals corresponding to the live event; 
a means for generating one or more graphics signals; 

at least one digital compression device, connected to the receiving and producing means, for 
digitally compressing the video, graphics and audio signals; 

a digital multiplexer, connected to the digital compression device, for multiplexing the video, 
graphics and audio signals, into a combined digital program stream; and 

a means for transmitting the combined digital program stream. 

6. The system of claim 5, wherein the transmission means is a satellite transmission system. 

7. The system of claim 5, wherein the transmission means is a cable distribution system. 

8. The system of claim 5, wherein the transmission means is a broadcast transmission system. 
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9. The system of claim 5, wherein the combined digital program stream is received within a private 
network. 

10. The system of claim 5, wherein the combined digital program stream is received within an in- 
stadimn network. 

1 1. The system of claim 5, wherein the combined digital program stream is received over the 
Internet. 
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ABSTRACT 

The present invention relates to an interactive digital system enabling viewers full and active 
participation in experiencing a live broadcast event. Particularly, the presentation of the live event is 
personaUzed for the viewer through the provision of various options, including multiple video streams, 
associated with different camera angles, for example, and integrated audio and graphics segments. 
Further, information obtained from related Web sites can be integrated into the live program. Various 
video and audio streams are collected from a live event and forwarded to a central control studio. 
Graphics are created at the central studio on a personal computer or chyron device. After receiving the 
video, audio and graphics signals, the signals are digitized and compressed in digital compressors. 
These signals are then combined with special data codes into a "digital package," and subsequently, 
transmitted over a cable distribution system. Once received at a viewer home, the signals are received 
and processed in an interactive digital cable box. Selections of the video, audio, graphics displays 
and/or Web pages can be made as a function of immediate viewer entries, or to interrogatory 
responses presented at the beginning or during the program, or based on a prestored viewer profile. 
Once a decision is made to switch from one video to another video option, the digital switch is 
performed seamlessly. The digital interactive system is based upon seamless branches which occur in 
the course of full-motion video. 
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